Fine-grained lineage for safer notebook interactions
نویسندگان
چکیده
Computational notebooks have emerged as the platform of choice for data science and analytical workflows, enabling rapid iteration exploration. By keeping intermediate program state in memory segmenting units execution into so-called "cells", allow users to enjoy particularly tight feedback. However, cells are added, removed, reordered, rerun, this hidden accumulates, making behavior difficult reason about, leading errors lack reproducibility. We present nbsafety, a custom Jupyter kernel that uses runtime tracing static analysis automatically manage lineage associated with cell global notebook state. nbsafety detects prevents make during unaided interactions, all while preserving flexibility existing semantics. evaluate nbsafety's ability prevent erroneous interactions by replaying analyzing 666 real sessions. Of these, identified 117 sessions potential safety errors, remaining 549 sessions, resolving issues were more than 7X likely be selected re-execution compared random baseline, even though not using therefore influenced its suggestions.
منابع مشابه
Smoke: Fine-grained Lineage at Interactive Speed
Data lineage describes the relationship between individual input and output data items of a workflow, and has served as an integral ingredient for both traditional (e.g., debugging, auditing, data integration, and security) and emergent (e.g., interactive visualizations, iterative analytics, explanations, and cleaning) applications. The core, long-standing problem that lineage systems need to a...
متن کاملSupporting Fine-grained Data Lineage in a Database Visualization Environment
The lineage of a datum records its processing history. Because such information can be used to trace the source of anomalies and errors in processed data sets, it is valuable to users for a variety of applications including investigation of anomalies and debugging. Traditional data lineage approaches rely on metadata. However, metadata does not scale well to fine-grained lineage, especially in ...
متن کاملUltra-Fine Grained Dual-Phase Steels
This paper provides an overview on obtaining low-carbon ultra-fine grained dual-phase steels through rapid intercritical annealing of cold-rolled sheet as improved materials for automotive applications. A laboratory processing route was designed that involves cold-rolling of a tempered martensite structure followed by a second tempering step to produce a fine grained aggregate of ferrite and ca...
متن کاملSpatio-Temporal Detection of Fine-Grained Dyadic Human Interactions
We introduce a novel spatio-temporal deformable part model for offline detection of fine-grained interactions in video. One novelty of the model is that part detectors model the interacting individuals in a single graph that can contain different combinations of feature descriptors. This allows us to use both body pose and movement to model the coordination between two people in space and time....
متن کاملAttention for Fine-Grained Categorization
This paper presents experiments extending the work of Ba et al. (2014) on recurrent neural models for attention into less constrained visual environments, beginning with fine-grained categorization on the Stanford Dogs data set. In this work we use an RNN of the same structure but substitute a more powerful visual network and perform large-scale pre-training of the visual network outside of the...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the VLDB Endowment
سال: 2021
ISSN: ['2150-8097']
DOI: https://doi.org/10.14778/3447689.3447712